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CLAIMS 



1. One or more computer-readable media having stored thereon a 
pluraHty of instructions that, when executed by one or more processors of a 
computer, causes the one or more processors to perform the following acts: 

receiving search criteria; 

generating a query vector based on text features of the search criteria; 

identifying media content pieces to be rendered by comparing the query 
vector to text feature vectors associated with a plurality of media content pieces; 

receiving user feedback regarding the relevancy of the identified media 
content pieces; ' 

modifying the query vector based on the user feedback; 

modifying one or more of the text feature vectors associated with the 
plurality of media content pieces based on the user feedback; and 

identifying new media content pieces to be rendered by comparing the 
modified query vector to the text feature vectors, including the one or more 
modified text feature vectors, associated with the plurality of media content 
pieces. 

2. One or more computer readable media as recited in claim 1, further 
comprising: 

generating another query vector based on one or more low-level features of 
the search criteria; and 

wherein the identifying comprises. 
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• # 

comparing the query vector to text feature vectors associated with 
the pluraUty of media content pieces to generate first resuhs, 

comparing the other query vector to other low-level feature vectors 
associated with the plurality of media content pieces to generate second 
results, and 

combining, for one of the plurality of media content pieces, the first 
and second results corresponding to the one media content piece. 

3. One or more computer readable media as recited in claim 2, further 
comprising altering, based on the user feedback, a weighting of the results used in 
the combining. 

4. One or more computer readable media as recited in claim 3, wherein 
the altering comprises: 

determining, for the one of the plurality of media content pieces, whether 
the first result corresponding to the one media content piece is greater than the 
second result corresponding to the one media content piece; and 

weighting the first result corresponding to the one media content piece 
more heavily if the first result corresponding to the one media content piece is less 
than the second result corresponding to the one media content piece, and otherwise 
weighting the second result corresponding to the one media content piece more 
heavily. 



Lee <&. Hayes. PLLC 



34 



MS1-725US.PA T.APRDOC 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 




5. One or more computer readable media as recited in claim 1 , wherein 
modifying one or more of the text feature vectors associated with the plurality of 
media content pieces based on the user feedback comprises altering a weighting of 
one or more elements in the feature vector based on the user feedback. 

6. One or more computer readable media as recited in claim 1 , wherein 
the search criteria comprises one or more words. 

7. One or more computer readable media as recited in claim 1 , wherein 
the piece of media content comprises an image. 

8. One or more computer readable media as recited in claim 1 , wherein 
the piece of media content comprises a portion of audio content. 

9. One or more computer readable media as recited in claim 1 , wherein 
the piece of media content comprises a portion of multimedia content. 

10. A method comprising: 
identifying a media content source; 

collecting one or more pieces of media content and associated text from the 
media content source; 

extracting, for a piece of media content, one or more text features from the 
associated text; and 

making the one or more text features available for searching. 
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11. A method as recited in claim 10, further comprising: 

generating one or more text feature vectors from the extracted one or more 
text features; and 

wherein the making comprises making the one or more text feature vectors 
available for searching. 

12. A method as recited in claim 10, further comprising: 

extracting one or more low-level features from the media content piece; and 
making the one or more low-level features available for searching. , 

13. A method as recited in claim 10, further comprising, for each 
collected piece of media content: 

classifying the image as meaningful or not meaningful; and 

wherein the extracting comprises extracting the one or more text features 

for the piece of media content only if the piece of media content is classified as 

meaningful. 

14. A method as recited in claim 10, wherein the media content source 
comprises a web site including a plurality of web pages, each web page including 
a plurality of pieces of media content and text associated with one or more of the 
plurality of pieces of media content. 
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15. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a filename and the one or more text features 
comprises one or more words in the filename. 

16. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a uniform resource locator (URL) and the one or 
more text features comprises one or more words in the URL. 

17. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises altemate text that can be displayed in place of 
the media content, and the one or more text features comprises one or more words 
of the altemate text. 

18. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises text surrounding the piece of media content on a 
web page, and the one or more text features comprises one or more words of the 
text surrounding the piece of media content. 

19. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a title of a web page that includes the piece of 
media content, and the one or more text features comprises one or more words in 
the title. 
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20. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises a hnk on a web page that includes the piece of 
media content, and the one or more text features comprises one or more words in 
the link. 

21. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises anchor text corresponding to the piece of media 
content, and the one or more text features comprises one or more words in the 
anchor text. 

22. A method as recited in claim 10, wherein the associated text for a 
piece of media content comprises an image annotation corresponding to the piece 
of media content, and the one or more text features comprises one or more words 
in the image annotation. 

23. A method as recited in claim 10, wherein each of the one or more 
pieces of media content comprises an image. 

24. A method as recited in claim 10, wherein each of the one or more 
pieces of media content comprises a piece of audio content. 

25. A method as recited in claim 10, wherein each of the one or more 
pieces of media content comprises a piece of multimedia content. 
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26. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
10. 

27. A method comprising: 
receiving search criteria; 

generating a query vector based on the search criteria; 

comparing the query vector to a feature vector corresponding to a piece of 
media content and having been generated based on text associated with the piece 
of media content; and 

determining, based at least in part on a result of the comparing, whether to 
render the piece of media content to a user. 

28. A method as recited in claim 27, further comprising: 
rendering a plurality of pieces of media content to the user; 

receiving feedback from the user regarding one or more of the plurality of 
pieces of media content; 

modifying the query vector based on the feedback; and 

repeating the comparing and determining based on the modified query 

vector. 

29. A method as recited in claim 27, further comprising: 

generating another query vector based on one or more low-level features of 
the search criteria; 
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comparing the other query vector to another feature vector corresponding to 
the piece of media content and having been generated based on the one or more 
low-level features of the piece of media content; 

combining a result of the other query vector to the other feature vector with 
the result of comparing the query vector to the feature vector; and 

wherein the determining comprises determining, based at least in part on 
the combined result, whether to render the piece of media content to the user. 

30, A method as recited in claim 29, further comprising: 

accessing user feedback regarding the relevancy of one or more pieces of 
media content rendered to the user; and 

altering, based on the user feedback, a weighting of the results during the 
combining. 

31. A method as recited in claim 30, wherein the altering comprises: 
determining whether a first distance between the other query vector and the 

other feature vector is greater than a second distance between the query vector and 
the feature vector; and 

weighting the result of the comparing the other query vector to the other 
feature vector more heavily if the first distance is less than the second distance, 
and otherwise weighting the result of the comparing the query vector to the feature 
vector more heavily. 
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32. A method as recited in claim 27, further comprising: 

accessing user feedback regarding the relevancy of one or more pieces of 
media content rendered to the user; and 

modifying the feature vector corresponding to the piece of media content 
based on the user feedback. 

33. A method as recited in claim 32, further comprising: 

generating a user space vector corresponding to each of the one or more 
pieces of media content for which user feedback is accessed; and 

using the user space vector corresponding to a particular piece of media 
content to modify the feature vector corresponding to the particular piece of media 
content. 

34. A method as recited in claim 27, further comprising: 

accessing user feedback regarding the relevancy of one or more pieces of 
media content rendered to the user; and 

altering a weighting of one or more elements in the feature vector based on 
the user feedback. 

35. A method as recited in claim 27, wherein the search criteria 
comprises one or more words. 

36. A method as recited in claim 27, wherein the piece of media content 
comprises an image. 
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37. A method as recited in claim 27, wherein the piece of media content 
comprises a piece of audio content. 

38. A method as recited in claim 27, wherein the piece of media content 
comprises a piece of multimedia content. 

39. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
27. 

40. A system comprising: 

a crawler module coupled to access a media content source and collect a 
plurality of media content pieces and associated text from the media content 
source; 

a feature extraction module coupled to extract one or more text features 
from one of the media content pieces; and 

a media content indexing module coupled to generate a text feature vector, 
based on the extracted one or more text features, corresponding to the one media 
content piece. 

41. A system comprising: 

a query generator to generate a query vector based on received search 
criteria; and 

a matching module coupled to. 



Lee & Hayes, PLLC 



42 



MSJ-725US.PAT.APP.OOC 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



receive the query vector and compare the query vector to a plurality 
of feature vectors corresponding to a plurality of pieces of media content, 
wherein each of the plurality of feature vectors has been generated based on 
text associated with one of the plurality of pieces of media content, and 

identify one or more of the plurality of pieces of media content to 
retum for rendering to a user based on the comparison of the query vector 
to the plurality of feature vectors. 

42. A method comprising: 
receiving search criteria; 

identifying, based at least in part on the search criteria, a piece of media 
content to be rendered; 

receiving user feedback regarding the relevancy of the rendered piece of 
media content; 

weighting for another piece of media content, based on the user feedback, 
both a result of comparing the high-level query vector to a high-level feature 
vector of the other piece of media content and a result of comparing the low-level 
query vector to a low-level feature vector of the other piece of media content; and 

combining the weighted result to determine whether to identify the other 
piece of media content for rendering, 

43. A method as recited in claim 42, further comprising generating a 
new high-level query vector and a new low-level query vector based at least in 
part on the search criteria. 
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44. A method as recited in claim 42, further comprising: 

generating a user space vector corresponding to the piece of media content; 

and 

using the user space vector corresponding to the piece of media content to 
modify the high-level feature vector corresponding to the piece of media content. 

45. A method as recited in claim 42, further comprising altering a 
weighting of one or more elements in the feature vector based on the user 
feedback. 

46. A method as recited in claim 42, wherein the high-level feature 
vector of the other piece of media content is a text feature vector. 

47. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
42. 

48. A method comprising: 

receiving user feedback regarding the relevancy of one or more pieces of 
rendered media content; and 

for each of the one or more pieces of media content, 

modifying, based on the user feedback, a feature vector 
corresponding to the piece of media content, and 

making the modified feature vector available for subsequent 
searching of the one or more pieces of media content. 
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49. A method as recited in claim 48, wherein the feature vector is a text 
feature vector. 

50. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
48. 

51. One or more computer-readable media having stored thereon a 
plurality of instructions that, when executed by one or more processors of a 
computer, causes the one or more processors to perform acts including: 

identifying a piece of media content to render to a user based at least in part 
on comparing a query vector corresponding to search criteria of the user and a 
feature vector corresponding to the piece of media content; 

receiving user feedback regarding the relevancy of the piece of media 
content; 

modifying the query vector based on the received user feedback; and 
modifying the feature vector based on the received user feedback in an off- 
line log mining process. 

52. One or more computer-readable media as recited in claim 51, 
wherein modifying the query vector comprises generating a vector U based on 
pieces of media content identified as relevant in the user feedback, and generating 
a new query vector D^^^ according to the following: 

D„^=7iU + {\-7j)D 
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where r] represents a confidence in the vector U . 

53, One or more computer-readable media as recited in claim 52, 
wherein modifying the query vector comprises generating a vector V based on 
pieces of media content identified as irrelevant in the user feedback, and 
generating a new query vector Z)^^^/ according to the following: 

54. One or more computer-readable media as recited in claim 51, 
wherein the piece of media content comprises one of: audio content, visual 
content, and multimedia content. 
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